Análisis de tweets de @MamelaFialloFlo

Información

Los datos de este usuario cubren desde la creación de la cuenta 2011-06-15 hasta 2023-03-01

Code

df.info()

<class 'pandas.core.frame.DataFrame'>
Index: 23687 entries, 21498 to 45184
Data columns (total 63 columns):
 #   Column                   Non-Null Count  Dtype                            
---  ------                   --------------  -----                            
 0   query                    23687 non-null  object                           
 1   id                       23687 non-null  float64                          
 2   timestamp_utc            23687 non-null  int64                            
 3   local_time               23687 non-null  object                           
 4   user_screen_name         23687 non-null  object                           
 5   text                     23687 non-null  object                           
 6   possibly_sensitive       4337 non-null   object                           
 7   retweet_count            23687 non-null  float64                          
 8   like_count               23687 non-null  float64                          
 9   reply_count              23687 non-null  float64                          
 10  impression_count         1985 non-null   object                           
 11  lang                     23687 non-null  object                           
 12  to_username              16705 non-null  object                           
 13  to_userid                16705 non-null  float64                          
 14  to_tweetid               16687 non-null  float64                          
 15  source_name              23687 non-null  object                           
 16  source_url               23687 non-null  object                           
 17  user_location            0 non-null      object                           
 18  lat                      0 non-null      object                           
 19  lng                      0 non-null      object                           
 20  user_id                  23687 non-null  object                           
 21  user_name                23687 non-null  object                           
 22  user_verified            23687 non-null  float64                          
 23  user_description         23687 non-null  object                           
 24  user_url                 0 non-null      object                           
 25  user_image               23687 non-null  object                           
 26  user_tweets              23687 non-null  object                           
 27  user_followers           23687 non-null  float64                          
 28  user_friends             23687 non-null  object                           
 29  user_likes               23687 non-null  float64                          
 30  user_lists               23687 non-null  float64                          
 31  user_created_at          23687 non-null  object                           
 32  user_timestamp_utc       23687 non-null  float64                          
 33  collected_via            23687 non-null  object                           
 34  match_query              23687 non-null  float64                          
 35  retweeted_id             0 non-null      float64                          
 36  retweeted_user           0 non-null      float64                          
 37  retweeted_user_id        0 non-null      float64                          
 38  retweeted_timestamp_utc  0 non-null      object                           
 39  quoted_id                2119 non-null   object                           
 40  quoted_user              2119 non-null   object                           
 41  quoted_user_id           2119 non-null   float64                          
 42  quoted_timestamp_utc     2119 non-null   float64                          
 43  collection_time          23687 non-null  object                           
 44  url                      23687 non-null  object                           
 45  place_country_code       58 non-null     object                           
 46  place_name               58 non-null     object                           
 47  place_type               58 non-null     object                           
 48  place_coordinates        58 non-null     object                           
 49  links                    1626 non-null   object                           
 50  domains                  1626 non-null   object                           
 51  media_urls               3806 non-null   object                           
 52  media_files              3806 non-null   object                           
 53  media_types              3806 non-null   object                           
 54  media_alt_texts          361 non-null    object                           
 55  mentioned_names          17305 non-null  object                           
 56  mentioned_ids            16211 non-null  object                           
 57  hashtags                 2499 non-null   object                           
 58  intervention_type        0 non-null      float64                          
 59  intervention_text        0 non-null      float64                          
 60  intervention_url         0 non-null      float64                          
 61  country                  23687 non-null  object                           
 62  date                     23687 non-null  datetime64[ns, America/Guayaquil]
dtypes: datetime64[ns, America/Guayaquil](1), float64(20), int64(1), object(41)
memory usage: 11.6+ MB

Datos

Code

df.head(3)

	query	id	timestamp_utc	local_time	user_screen_name	text	possibly_sensitive	retweet_count	like_count	reply_count	...	media_types	media_alt_texts	mentioned_names	mentioned_ids	hashtags	intervention_type	intervention_text	intervention_url	country	date
21498	from:MamelaFialloFlo	1.638177e+18	1679406841	2023-03-21T13:54:01	MamelaFialloFlo	Hoy en Ciudad de México 🇲🇽se presenta España l...	0.0	25.0	78.0	4.0	...	photo|photo	|	NaN	NaN	NaN	NaN	NaN	NaN	Ecuador	2023-03-21 08:54:01-05:00
21499	from:MamelaFialloFlo	1.637879e+18	1679335694	2023-03-20T18:08:14	MamelaFialloFlo	No podemos defender a la mujer si le decimos a...	NaN	79.0	314.0	17.0	...	video	NaN	NaN	NaN	lgbt	NaN	NaN	NaN	Ecuador	2023-03-20 13:08:14-05:00
21500	from:MamelaFialloFlo	1.637875e+18	1679334914	2023-03-20T17:55:14	MamelaFialloFlo	@Xileone Gracias por destacarlo.	NaN	0.0	9.0	0.0	...	NaN	NaN	xileone	65372761	NaN	NaN	NaN	NaN	Ecuador	2023-03-20 12:55:14-05:00

3 rows × 63 columns

Dominios

Lista del top 20 de dominios mencionados en los tweets y su frecuencia:

Code

# count items on column
domains = df['domains'].value_counts()

# return first n rows in descending order
top_domains = domains.nlargest(20)

top_domains

domains
panampost.com                321
es.panampost.com             165
youtube.com                  105
youtu.be                      85
twitter.com                   52
bit.ly                        43
instagram.com                 37
gaceta.es                     32
facebook.com                  21
buff.ly                       20
publichealth.lacounty.gov     19
eluniverso.com                18
amazon.com                    14
abc.es                        13
lozierinstitute.org           13
vatican.va                    10
amp.milenio.com                9
lifenews.com                   9
library.brown.edu              9
bbc.com                        9
Name: count, dtype: int64

Hashtags

Lista del top 20 de hashtags más usados y su frecuencia

Code

# convert dataframe column to list
hashtags = df['hashtags'].to_list()

# remove nan items from list
hashtags = [x for x in hashtags if not pd.isna(x)]

# split items into a list based on a delimiter
hashtags = [x.split('|') for x in hashtags]

# flatten list of lists
hashtags = [item for sublist in hashtags for item in sublist]

# count items on list
hashtags_count = pd.Series(hashtags).value_counts()

# return first n rows in descending order
top_hashtags = hashtags_count.nlargest(20)

top_hashtags

leyabortistano              243
femeninasífeministano        87
coronavirus                  87
salvemoslasdosvidas          64
blacklivesmatter             64
ecuadoresprovida             47
vetopresidencial             42
tiraníasanitaria             41
provida                      33
leydelviolador               29
síalavida                    28
noalaborto                   27
nohablesenminombre           26
abortolegal                  26
guateesvida                  24
datomatarelato               24
laviolencianotienegénero     23
covid19                      22
8m                           21
justiciaporlucio             19
Name: count, dtype: int64

Usuarios

Top 20 de usuarios más mencionados en los tweets

Code

# filter column from dataframe
users = df['mentioned_names'].to_list()

# remove nan items from list
users = [x for x in users if not pd.isna(x)]

# split items into a list based on a delimiter
users = [x.split('|') for x in users]

# flatten list of lists
users = [item for sublist in users for item in sublist]

# count items on list
users_count = pd.Series(users).value_counts()

# return first n rows in descending order
top_users = users_count.nlargest(20)

top_users

agustinlaje        330
lassoguillermo     324
mamelafialloflo    271
panampost_es       239
jairbolsonaro      210
etorrescobo        203
felipeleon88       177
realdonaldtrump    171
xileone            138
pjavieror          135
vox_es             131
asambleaecuador    129
simpliciterpaco    127
fundlibre          118
pmunoziturrieta    110
gloriaalvarez85    109
freityt            109
jmilei              98
avelinaponceg       97
pontifex_es         95
Name: count, dtype: int64

Likes en el tiempo

Code

# plot the data using plotly
fig = px.line(df, 
              x='date', 
              y='like_count', 
              title='Número de likes en el tiempo',
              template='plotly_white', 
              hover_data=['text'])

# show the plot
fig.show()

Tokens

Lista del top 20 de los tokens más comunes y su frecuencia

Code

# load the spacy model for Spanish
nlp = spacy.load("es_core_news_sm")

# load stop words for Spanish
STOP_WORDS = nlp.Defaults.stop_words

# Function to filter stop words
def filter_stopwords(text):
    # lower text
    doc = nlp(text.lower())
    # filter tokens
    tokens = [token.text for token in doc if not token.is_stop and token.text not in STOP_WORDS and token.is_alpha]
    return ' '.join(tokens)

# apply function to dataframe column
df['text_pre'] = df['text'].apply(filter_stopwords)

# count items on column
token_counts = df["text_pre"].str.split(expand=True).stack().value_counts()[:20]

token_counts

gracias       2344
mujer         1729
vida          1548
mujeres       1394
aborto        1381
feminismo     1167
libertad       924
matar          881
derecho        701
the            666
ecuador        657
hombre         630
quieren        617
feministas     614
izquierda      603
madre          586
personas       549
dios           529
violencia      528
hombres        528
Name: count, dtype: int64

Hora

Lista de las 10 horas con más cantidad de tweets publicados

Code

# extract hour from datetime column
df['hour'] = df['date'].dt.strftime('%H')

# count items on column
hours_count = df['hour'].value_counts()

# return first n rows in descending order
top_hours = hours_count.nlargest(10)

top_hours

hour
08    1665
09    1659
10    1573
22    1571
07    1522
11    1338
23    1312
21    1302
19    1268
12    1237
Name: count, dtype: int64

Pataformas

Plataformas desde las que se publicaron contenidos y su frecuencia

Code

df['source_name'].value_counts()

source_name
Twitter for iPhone     16202
Twitter Web App         7273
Twitter Web Client       165
Twitter for Android       47
Name: count, dtype: int64

Tópicos

Técnica de modelado de tópicos con transformers y TF-IDF

Code

# visualize topics
topic_model.visualize_topics()

Reducción de tópicos

Mapa con 10 tópicos del contenido de los tweets

Code

# visualize topics
topic_model.visualize_topics()

Términos por tópico

Code

topic_model.visualize_barchart(top_n_topics=11)

Análisis de tópicos

Selección de tópicos que tocan temas de género

Code

# selection of topics
topics = [0]

keywords_list = []
for topic_ in topics:
    topic = topic_model.get_topic(topic_)
    keywords = [x[0] for x in topic]
    keywords_list.append(keywords)

# flatten list of lists
word_list = [item for sublist in keywords_list for item in sublist]

# use apply method with lambda function to filter rows
filtered_df = df[df['text_pre'].apply(lambda x: any(word in x for word in word_list))]

percentage = round(100 * len(filtered_df) / len(df), 2)
print(f"Del total de {len(df)} tweets de @MamelaFialloFlo, alrededor de {len(filtered_df)} hablan sobre temas de género, es decir, cerca del {percentage}%")

Del total de 23687 tweets de @MamelaFialloFlo, 8110 hablan sobre estos temas
Alrededor del 34.24% de los tweets de @MamelaFialloFlo hablan sobre estos temas

Code

# drop rows with 0 values in two columns
filtered_df = filtered_df[(filtered_df.like_count != 0) & (filtered_df.retweet_count != 0)]

# add a new column with the sum of two columns
filtered_df['impressions'] = (filtered_df['like_count'] + filtered_df['retweet_count'])/2

# extract year from datetime column
filtered_df['year'] = filtered_df['date'].dt.year

# remove urls, mentions, hashtags and numbers
p.set_options(p.OPT.URL)
filtered_df['tweet_text'] = filtered_df['text'].apply(lambda x: p.clean(x))

# Create scatter plot
fig = px.scatter(filtered_df, x='like_count', 
                 y='retweet_count',
                 size='impressions', 
                 color='year',
                 hover_name='tweet_text')

# Update title and axis labels
fig.update_layout(
    title='Tweets talking about gender with most Likes and Retweets',
    xaxis_title='Number of Likes',
    yaxis_title='Number of Retweets'
)

fig.show()

Tópicos en el tiempo

Code

# convert column to list
tweets = df['text_pre'].to_list()
timestamps = df['local_time'].to_list()

topics_over_time = topic_model.topics_over_time(docs=tweets, 
                                                timestamps=timestamps, 
                                                global_tuning=True, 
                                                evolution_tuning=True, 
                                                nr_bins=20)

topic_model.visualize_topics_over_time(topics_over_time, top_n_topics=20)